Transductive Learning via Model Selection; Can Overfitting be Exploited?

A novel transductive learning algorithm is proposed, which is based on the use of model selection. In its simplest form there are k possible labels, m labeled points and one unlabeled point. One model is built for each possible classification of the unlabeled point yM+1 = Li, i = 1, ..., k, using all m+1 points and m + 1 labels. Any standard model selection criterion can then be applied to select one of the k models. The algorithm simply chooses the label Li that produced that model. We define the algorithm, show statistical justifications for it, and experimentally show the effectiveness of the algorithm when using a simple linear model combined with a model selection based on cross-validation.